How to transform correlated random variables into uncorrelated ones

K e y w o r d s R e p a r a m e t r i z a t i o n , One-to-one transformation, Monotone function covariance. 1. I N T R O D U C T I O N If X is an a r b i t r a r y n -d imens iona l r a n d o m vector , t hen in m a n y s t a t i s t i ca l p rob lems , it is useful to cons t ruc t unco r r e l a t ed r a n d o m variables , Y1 ,Y2 , . . . , Yn such t h a t X = f ( Y ) , where Y = (Y1, Y 2 , . . . , yn) . T h e n f can be a l inear funct ion, i.e., X = A Y , where A is a ma t r ix . Unfo r tuna te ly , th is s imple l inear funct ion, if it is invert ible: Y = A i X might c omp le t e ly des t roy t h e or ig ina l mean ing of the X var iables by forming the i r " l inear mix tures" in o rde r to make t h e m uncor re la ted . If we do not wan t to mix up un re l a t ed quant i t i es , t hen i t seems to be i m p o r t a n t to find funct ions gl, g2 , . . . , gn t h a t make g l (X1) , g2(X2) . . . . , gn(Xn) uncor re l a t ed . As a first s tep, we prove the ex is tence of R --* R funct ions f l , f 2 , . . -, fn such t h a t (X1, X 2 , . . . , X~) = ( f l (Y1), f2 (Y2) ,. • . , fin (Y~)), where ]I1, Y 2 , . . , Yn are uncor re la ted . If the f s t u r n out to be oneto-one , t hen gi = f.:~l solves our p rob lem. Even w i th in the class of oneto-one f s , it can be in te res t ing to find as s imple ones a s is poss ib le (e.g., monotone , piecewise l inear, e tc .) . In th is pape r , we solve some of these p rob lems . U n c o r r e l a t e d (or thogona l ) r e p a r a m e t r i z a t i o n s were cons idered from o the r po in t s of view by m a n y au tho r s , see, e.g., [1-3]. 0893-9659/00/$ see front matter (~) 2000 Elsevier Science Ltd. All rights reserved. Typeset by A.~g-TEX PII: S0893-9659 (00)00050-1 32 A.K. GUPTA et al. 2. A G E N E R A L R E S U L T THEOREM 1. For an arbitrary random vector X = ( X 1 , X 2 , . . . , X n ) , one can always And another random vector Y = (]I1, Y2 , . . . , Yn) with uncorrelated components and R -~ R functions f l , f 2 , . . . , fn such that (X1, X 2 , . . . , X~) = (f l (Y1), f2 (Y2) . . . . , f n ( Y n ) ) . PROOF. Without loss of generality, we may suppose that 0 < Xi < 1, i = 1, 2 , . . . , n (otherwise, apply the one-to-one transformation X~ = 1/2 + (1/rr)arctanX~). We are going to prove tha t there exist indicator random variables I1, I 2 , . . . , I,~ such that I = I1, I 2 , . . . , In) is independent of X and Y i = X i + c l i , i = 1 , 2 , . . . , n are uncorrelated (if c is suitably chosen) and Yi uniquely determines Xi (and Yi). Denote by E = (a~j) the covariance matr ix of X. If (N1, N 2 , . . . , N~) is an n-variate normal vector with correlation matr ix R = (r~j) and Ii = sign Ni, then coy ( i~ , I j ) = P(N~ > O,N~ > O) 1 1 -arcsin rij. 4 27r I f c is big enough, then the numbers rij = -2rrsinc~ij/e form a correlation matr ix (R = (rij) is positive definite) and with these correlations rij, we have coy ( ~ , ~ ) = c o v ( x i , x j ) + c 2 cov ( I . I j ) = 0 We can easily reconstruct Xi (and Ii) from Yi (this is where we use the restriction 0 < Xi <_ 1): Xi = Yi c[YJc] where [] denotes the integer part. REMARK 1. In this construction above, fi (i = 1 ,2 , . . . ,n) is not one-to-one, since for a given value of Xi, there may correspond two different Yi values, either Yi = Xi or Yi = X.i + c. We cannot even hope tha t there always exist one-to-one functions fi with which Theorem 1 holds. For example, if n = 2, and both X1 and )(2 take only two different values, then all functions on these two points are linear, but linear one-to-one functions of correlated X1 and X2 will not become uncorrelated. REMARK 2. If in Theorem 1, we need to choose Y independently of X (only the f s depend on X), then it is clear that the maximal correlation of Yi, Yj, i , j = 1 , 2 , . . . , n must be 1 (otherwise, Theorem 1 cannot hold for X variables with maximal correlation 1). Using a oneto-one t ransformation between R n on [0, 1), we can easily see tha t every random vector X = (X1, X 2 , . . . , Xn) has the following representation: ( X l , X 2 , . . . , X n ) : ( g l ( U ) , g 2 ( U ) , . . . , g n ( U ) ) , where U is uniformly distributed on [0, 1). Finally, from the proof of Theorem 1, we can see that Y i = U + c I i , i = 1 , 2 , . . . , n becomes uncorrelated with suitable choices of c and I = (I1, I 2 , . . . , I n ) . This special choice of uncorrelated Y1, Y2, . . . , Yn variables (with maximal correlation 1) is "universal" (does not depend on X). The "price" is tha t the f s become more complicated. Correlated Random Variables 33 3. H O W SIMPLE C A N T H E fs BE? As we pointed out in Remark 1, the functions f in Theorem 1 are not always one-to-one. What we can say of X is "very nice", e.g., if X is normal. Suppose for simplicity tha t n = 2. THEOREM 2. If (Xi , X2) is bivariate normal with nonzero correlation r, then no monotone timctions f i , f2 can make f l ( X i ) and f2(X2) uneorrelated. On the other hand, there always exist one-to-one piecewise linear functions that make them uncorrelated. I f EX1 = EX2 = O, then a simple choice is X, for ]X] > ecr2, f l ( X ) = X, f 2 ( X ) = x , *'or IXl < c0"2, where e does not depend on either Far Xi or r (c is "universal" for all bivariate normal variables with r ¢ 0). PROOF. We may suppose that r > 0. Then Xi and X2 are positively quadrant dependent (see [4]), i.e., A(x, y) = Fxl,x2 (x, y) Fx, (x)Fx2 (y) > 0, for all real x and y. Using Hoeffding's covariance formulas for f i (X1) and f2(X2) for monotone f l and f2, we get o P (Xi ) , f2 (X2)) = JJ =(~,v) dfl(x) ( f l df2(y), COV if Var f~(Xi) < oc. Since A(x, y) > 0, this covariance cannot be 0 for monotone f i and f2. Denote by ¢(x) the probability density function of the standard normal random variable, r h e a t, he special piecewise linear form of f l and f2 in the theorem gives COV()el ( X l ) , f 2 (X2) ) = E ( f 2 ( X 2 ) Z ( X 1 I X2) ) F = 7"0"10" 2 -X2¢(X) d x + 2